Skip to content

fix(system): honor scan.snapshot-id in metadata system tables - #498

Merged
lxy-9602 merged 1 commit into
apache:mainfrom
fastio:feat/system-table-pinned-snapshot
Nov 4, 2026
Merged

lxy-9602 merged 1 commit into
apache:mainfrom
fastio:feat/system-table-pinned-snapshot

Conversation

@fastio

@fastio fastio commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #497

FilesSystemTable and ManifestsSystemTable always read the latest snapshot, even when the
caller sets scan.snapshot-id. A reader that pins a snapshot therefore gets file and manifest
metadata from a newer commit, and a snapshot id that does not exist silently falls back to the
latest snapshot.

This change replaces LatestSnapshot() with SelectedSnapshot(), which reads
CoreOptions::GetScanSnapshotId():

  • unset: the latest snapshot, as before;
  • set and present: that snapshot;
  • set but missing: Status::NotExist("snapshot <id> does not exist") instead of a fallback.

ReadLatestManifestEntries / ReadLatestDataFiles are renamed to
ReadSelectedManifestEntries / ReadSelectedDataFiles to match. ManifestsSystemTable now
builds CoreOptions before selecting the snapshot.

Other time travel options (scan.tag-name, scan.timestamp-millis, ...) are out of scope.
Tables that list history ($snapshots, $schemas, $tags, ...) are unchanged.

Tests

  • SystemTableReadInteTest.TestReadMetadataSystemTablesAtPinnedSnapshot: commits two
    snapshots to a primary-key table, then checks that
    • without the option, $files returns both files;
    • with scan.snapshot-id = 1, $files returns only the first file (record_count = 1) and
      $manifests returns fewer manifests than the latest snapshot;
    • with scan.snapshot-id = 3, both tables fail with snapshot 3 does not exist.

Verified on Ubuntu 24.04 (aarch64), GCC 13, Debug: paimon-read-inte-test passes (303 tests),
including the 31 SystemTableReadInteTest cases.

API and Format

No public API change (only an internal file under src/). No storage format or protocol change.

Documentation

No.

Generative AI tooling

Generated-by: Claude Code (Claude Opus 5.5)

@lxy-9602 lxy-9602 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602
lxy-9602 merged commit da88188 into apache:main Nov 4, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] $files and $manifests system tables ignore scan.snapshot-id

2 participants